1. Technical Field
The present disclosure related generally to conversion of wide-angle images into rectilinear images.
2. Background Information
A diagrammatic programming language is one in which visual expressions, and the interconnections among visual expressions, are used to control program functions. Rather than simply enter text commands, a user manipulates the visual expressions in an interactive manner, according to a program structure and grammar. Many diagrammatic programming languages are structured according to a block diagram, where a set of nodes, called blocks, are interconnected by lines that represents signals. Blocks are functional entities that perform mathematical operations, transformations, or both, on data and information passed through the block diagram. Signals are streams of data and information that that flow between various blocks. Signals have particular data types, and may also include timing and control information related to the blocks.
One language that employs a block diagram, with blocks interconnected by signals, is the Simulink® technical computing environment available from The Mathworks, Inc. of Natick, Mass. The Simulink® environment allows a user to create a dynamic system by constructing a block diagram from a plurality of blocks provided in a pre-defined library, or specially defined by a user. Each block produces outputs either continuously (i.e. a continuous block) or at specific points in time (i.e. a discrete block). The signals transmit the outputs to other blocks to indicate interactions between the blocks.
With the Simulink® environment, a user may simulate (model) the behavior of a system in response to certain input and control data. Such a simulation may be performed in the diagrammatic programming environment itself, using an interpretive process. Alternately, a user may convert the block diagram to executable code. Automatic code generation is a process where text-based program code (such as C or C++ code) is automatically produced from a diagrammatic representation. The text-based program code is then compiled to produce an executable file. In this manner, the functionality specified in the block diagram may be exported beyond the diagrammatic programming environment to computers that lack the ability to interpret block diagrams. In yet another alternative, the block diagram may be translated to code specific for an embedded hardware target, such as a digital signal processing (DSP) chip, and this code used with the embedded hardware target to execute the functionality of the block diagram. In this manner, a block diagram may be used in rapid prototyping and testing applications, where hardware devices configured in a particular manner are required.
One area where the above described diagrammatic programming techniques may be employed, is in the field of image manipulation. Historically, most image manipulation was performed by optical lenses attached to cameras or other optical sensors (collectively image acquisition devices). For example, if magnification of an image was desired, an optical zoom having a mechanical assembly of lenses was employed with an image acquisition device. Similarly, if a different field of view (i.e. an angle of view defining a region of the environment captured) was desired, a lens adapted to capture that particular field of view was physically mounted to the image acquisition device.
Currently, it is often more economical to perform image manipulation in software, rather than with optics, due to the availability of relatively inexpensive computing resources. Further, software allows for many types of advanced image manipulation that are impossible using only optical lenses. Accordingly, it is now common to acquire images in a first format and then extensively process the images to create images of a second format that is more desirable to a user.
One format in which an image may be captured is the wide-angle format. A wide-angle lens is a lens configured with a decreased focal length, such that it captures a wider field of view than more conventional lenses. A wider field of view allows more of an environment to be captured in a given image, which may be beneficial for many applications. For example, in a security application where camera coverage of a given room is desired, use of a wide-angle lens may allow the coverage to be achieved with fewer cameras.
Yet, to achieve a wider field of view, a wide-angle lens generally must introduce perspective distortion into the image, causing the portion of the image that is located at the center to appear abnormally enlarged, and portions of the image towards the edges to appear abnormally compressed. Such distortion gives wide-angle images a distinctive convex appearance that is distracting to many viewers.
One particular type of wide-angle image is a Fish-Eye image, where the field of view has been extended to the extreme. A Fish-Eye image typically encompasses about a 180-degree (i.e., hemispherical) field of view, though some Fish-Eye images encompass a field of view that is somewhat greater or lesser in angle. Indeed, some known specialty Fish-Eye lenses produce images with a 220-degree field of view, while other known Fish-Eye lenses produce images with fields of view of only 150-degrees. A Fish-Eye image achieves its extremely wide field of view at the trade off of perspective distortion, which is typically severe. Thus, Fish-Eye images are typically characterized as curvilinear, where straight lines in the environment are represented as distinctly curved lines in the image. Despite this distortion, the benefits of Fish-Eye images make them desirable in a variety application. Fish-Eye lenses and the images they produce are particularly suited for a variety of applications in the fields of security, automotive, virtual reality, virtual tours, and scientific research.
Sometimes, it may be desirable to manipulate a Fish-Eye or other wide-angle image to remove some of the perspective distortion in a region of the image, and create a rectilinear image from the region. A rectilinear image is one where vertical, horizontal, and diagonal lines in the environment are displayed as substantially straight lines in the image. In order to represent these lines as substantially straight, a rectilinear image is restricted to a significantly smaller field of view than a Fish-Eye image. Yet, since rectilinear images are generally perceived by viewers to be “normal” (i.e. “correct”) images, they are widely used in photography, cinema, and other applications. Accordingly, conversion of a region of a Fish-Eye or other wide-angle image to a rectilinear image is often desirable. Such a conversion is sometimes referred to as dewarping the image, though the term may have other connotations in different contexts.
Several techniques are known for converting a Fish-Eye or other wide-angle image to a rectilinear image, yet these known techniques have shortcomings. In particular, they typically lack an intuitive way to specify the particular region of the Fish-Eye or other wide-angle image from which to construct the rectilinear image. Often, prior techniques require one to specify the size and orientation of such a region in terms of a variety of obscure angles and lengths that have little meaning to those outside of the field of optics. Accordingly, such prior techniques are generally unsuitable for users that have general computer training, but lack special knowledge of optics.